One-shot/few-shot learning
The first one-shot learning paper dates back to 2006, but becomes more popular recently.
Concepts
training/validation/test categories: Training categories and test categories have no overlap
support(sample)/query(batch) set: In the testing stage, for each test category, we preserve some instances to form the support set and sample from the remaining instances to form the query set
C-way K-shot: The test set has C categories. For each test category, we preserve K instances as the support set
episode: Episode-based strategy used in the training stage to match the inference in the testing stage. First sample some categories and then sample the suppport/query set for each category
Methods
Metric based:
Siamese network: the earliest and simplest metric-learning based few-shot learning, standard verification problem.
Matching network: map a support set to a classification function p(y|x,S) (KNN or LSTM). For the LSTM version, there is another similar work using memory module.
Relation network: calculate the relation score for 1-shot, calculate the average of relation scores for k-shot
Prototypical network: compare with the prototype representations of each class. Each class can have more than one prototype representation. There are some other prototype-based methods [1] [2].
Optimization (gradient) based:
[MAML] (https://arxiv.org/pdf/1703.03400.pdf)
REPTILE) (an approximation of MAML)
Model based:
Generation based: generate more features for novel categories [1], [2]
Pretraind and fine-tune: use the whole meta-training set to learn feature extractor [1] [2] pretrain+MatchingNet [3]